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This interim report evaluates the achievement of 
seven objectives in the Francis H* Howell School District Innovative 
Program. The objectives of the program were teacher improvement in 
the areas of: (a) effective questioning, (b) the writing of 
behavioral objectives, (c) diagnosis and evaluation, (d) 
individualized instruction, (e) science processes, (f) social studies 
inquiry, and (g) student behavioral modification. Workshops were held 
on individualized instruction, behaviqral objectives, and science and 
social studies teaching. Pre* and posttests administered for these 
workshops were the basis of their evaluation. The other objectives 
were not the subject of workshops, and their evaluation is based on 
subjective evidence. According to the objective and s^ibjective data 
analyzed, all objectives were partially or totally achieved. (HHD) 
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FRANCIS HOWELL E-III SCHOOLS 
TITLE III, ESEA INSEJRVICE TEACHER TRAINING 
PROGRAM EVALUATION 



INTRODUCTION 

The 1971*72 ESEA Title III program was evaluated by an outside 
agency. Their computerized analysis and subsequent interpretation of the 
raw evaluation data which was submitted by the local project was infinitely 
more sophisticated than a local evaluation effort could have been* 

There are several drawbacks, however, to contracting with an 
outside agency for project evaluation. Last year the outside agency conduct- 
ing our evaluation lost some of the raw data which we had submitted to 
them. In addition they were dilatory in submitting the evaluation report 
to us, thereby, causing us to be late in sending it to the State Department 
of Education. The cost of employing an outside agency is also a factor. 
The lack of communications between the local agency and the contracted agency - 
is a potential source of misunderstanding and confusion. 

It is the opinion of the director of the Francis Howell ESEA 
Title III project that there are some advantages to an internal evaluation: 
A« Direct control over the entire evalxzation effort. 

B. Greatly reduced evaluation cost. 

C. Improved communi'^ations between those responsible for the evaluation 
effort. 

This evaluation, eil though it may not possess the sophistication 
of the 1971-72 evaluation, will reflect the efforts of the Jrancis Howell 



Wtle III staff to present an objective look at the results of the 
1972-73 project. Wo have attempted to adhere as closely as possible to the 
evaluation design presented in the continuation application for 1972-73. 
Where it was impossible for one reason or another to obtain objective 
evaluation data, an explanation, and perhaps a subjective comment or two. 
has been substituted. 
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Chapter I 
GENERAL EVALUATION DESIGN 

The evaluation consisted primarily of a comparison of teachers* 
terminal behavior with the standards set forth in the objectives. Post- 
tests were used as the primary measurement technique to determine achievement 
of the stated objectives, which specified the desired terminal behavior. 
Pre-tests were also used in some workshops to help establish the baseline 
data needed for comparisons and determination of changes in teacher behaviors 
resulting from the project. 

In the *'Individ\£alized Instxiiction'' workshop, teachers were asked 
to submit a plan for individualizing .instruction in one subject area. 
Observations in the teacher's classroom were made to determine the degree 
of correlation between the proposed plan and the actual teaching method 
and to rate the classroom using the Individualized Instruction Observation 
Check List. 

COLLECTION OF THE DATA 

The data contained in this evaluation study was collected from 
pre and post tests, teacher prepared instructional material and observation 
check lists. The pre-tests were administered just previous to participation 
in a workshop and post-tests were administered at the conclusion of each 
workshop. The classroom observations were made periodically throughout the 
school year. The analysis of teacher prepared curriculum guides was done 
in July of 1973. 
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ANALYSIS OF DATA 

The data obtainod from the eveiluation effort is presented in both 
tabular and graphical fornio The mean differences between pre and post scores 
are used as one indication of project success • Another indication of success 
is the percentage of teachers who, as a result of workshop activities, 
changed their behavior sufficiently to meet the criteria of success stated 
in the continuation application. 

THE INSTRUMENTS 

In addition to the pre and post tests, three instruments were 
developed to use in evaluating the data: 

lo Individualized instruction observation check list* 

2o Social studios curriculum guide check list. 

3. Science curriculum guide check list. 
CJopies of these instrutnents are contained in the appendix 
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ANALYSIS OP THE DATA 

I. BEHAVIORAL OBJECTIVES 

Behavioral objectives were taught both in context and as a separate 
entity during the 1972-73 project year. They were taught in context in 
Individualized Instruction, learning activity packets, social studies, and 
science workshops. They were taught separately in a workshop entitled 
Behavioral Objectives - Their use and aisuse. Ten teachers who were involved 
in one or more of these workshops were selected randomly to write a list of 
ten behavioral objectives prior to participation in a workshop and to write 
another list of ten behavioral objectives at the conclusion of the workshop 
or workshops, if they were enrolled in more than one. The first list of 
objectives were used sxs baseline data for determining the degree of change 
in the participating teachers' skill in writing behavioral objectives. 
^ The three criteria used in evaluating the two sets of behavioral objectives 

are: 

1. The student behavior must be observable. 

2. The conditions under vtoich the performance or product is to 
occ\xr must be stated. 

3. The criterion or standard of performance must be stated. 
Using these three criteria as a standard, a perfect score on each 

behavioral objective would be three. A total of thirty points would be 
possible for each set of ten behavioral objectives. The scores, both pre 
and post, for or.ch individual teacher and for each group of teachers are 
presented in the table on the following page. 
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Pre and Post Inservice Training Scores 
Teacher Written Behavioral Objectives 



Teacher 


Pre 


Ere 


Post 


Poet 


Saw 


Gain 


Number 


Score 


% 


£)Core 


% 


Score 














Gain 




1 


10 


33.3 


30 


100.0 


20 


66.7 


2 




"50.0 




100-0 


21 

ii 1 ill 


70.0 


5 


7 


23.3 


30 


100.0 


23 


76.7 


k 


3 


10.0 


29 


96.6 


26 


86.6 


5 


6 


20.0 


29 


96.6 


23 


76.6 


O 


5 


16.6 


30 


100.0 


25 


83.'* 


7 


7 


23.3 


29 


96.6 


22 


73.3 


8 


6 


20.0 


30 


100.0 


2tf 


8o.o 


9 


2 


6.6 


29 


96.6 


27 


90.0 


10 


8 


26.6 


20 


66.6 


12 


to.o 


Mean 














Scores 


6.30 


20.97 


28.6 


95.33 


22.3 


71^.33 



II. INDIVIDUALIZEa) INCTHOCTION 

A total of twenty two teachers 3participated in the Individualized 
Instruction workshop during 1972-73* Of this number eight were first tiioe 
participants and fourteen wei^ involved in the workshop for the second year. 

An observation check list was developed by Dr. Jane VdlhouTi 
the project director, and workshop participants. This check list was then 
used to determine the extent to which the workshop participants were effectively 
implementing an individualized program in at least one subject area in their 
classroom. A written implementation plan was also collected from each 
workshop participant and has been included in this evaluation report. The 
observation check list contains thirty three separate criteria which are 
considered to be descriptive of a thorou^y individualized instructional 
program. This evaluation recognizes that there are degrees of individual* 
ization in every classroom and most wotild probably meet some of the criteria 
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whereas only a few would meet them allt 

From the list of thirty three criteria contained in the observation 
check sheet, the e valuators set an arbitrary standard for establishing the 
level of acceptance for an individualized classroom^ Those teachers who 
were second year participants in the workshop were expected to meet 709^ 
of the criteria^ The level of acceptance was set at 509^ for first year 
workshop participants, 

Th'S results of the observations made in each teacher's classroom 
are presented in Tables* II and III. 



TABLE II 
Second Year Participants 

Teacher Scores on the Individualized Instaruction Observation Caieck List 



Teacher 
Number 


^ Section 




% of 

Total 

Pose. 


Criteria 
Met 


I 


II 


1 III 




V 




VII 


Total 


















Points 


1 


5 


if 


3 


5 


2 


2 


3 


2if 


729^ 


yes 


2 


5 


if 


3 


6 


0 


2 


if 


2if 


72^ 


yes 


3 


5 


if 


2 


6 


if 


3 


if 


28 


8it9S 


yes 


k 


if 


if 


2 


6 


1 


5 


3 


25 


7356 


yes 


5 


5 


if 


3 


5 


if- 


3 


if 


30 


90?$ 


yes 


6 


k 


if 


2 


3 


3 


5 


if 


25 


73^ 


yes 


7 


5 


if 


2 


5 


if 


3 


3 


26 


78^ 


yes 


8 


3 


2 


if 


5 


if 


5 


if 


27 


81^ 


yes 


9 


5 


3 


if 


if 


if 


6 


if 


30 


909^ 


yes 


10 


k 


if 


if 


6 


if 


if 


if 


30 


90^ 


yes 


11 


5 


if 


if 


6 


if 


6 


2 


31 


93% 


yes 


12 


3 


3 


if 


6 


1 


if 


if 


25 


73% 


yes 


13 


if 


3 


3 


3 


2 


6 


if 


25 


73% 


yes 


l^f 


3 


3 


3 

1 


if 


if 


6 


2 

L 


25 

1 > 


73% 


yes 


Mean raw data 


if.2 


3.5 


3.0 


5.0 


2.0 


if.2 


3.3 


26.7 


0 




Scores per cent 


85^ 


899^ 


7^ 


8if$^ 


73?^ 


73^ 


839^ 


81^ 



ERIC 



8 



TABLE III 
First Year Participants 

Teacher Scores on the Individualized Instruction Observation Check List 



Teacher 
Number 



1 

2 

3 
k 

3 
6 

7 
8 



Section 



k 
5 
5 
5 
k 

3 
3 
3 



II i III 



k 

3 
if 

3 

if 

3 
2 
if 



2 

3 

2 
0 
2 
1 
2 
2 



IV 



5 

2 
5 

2 
if 

5 
1 

3 



3 
0 
0 
0 
0 
0 

1 



VI VII 



3 

1 

2 
0 

3 
1 
6 
5 



if 

3 
if 
2 
2 
if 
if 
2 



Total 
Points 



23 
20 
22 
12 
19 
17 
18 
20 



5lS of 

Total 

Poss* 



Criteria 
Net 



69^ 

319« 
57^ 

661^ 



yes 
yes 
yes 
no 
yes 
yes 
yes 
yes 



Mean raw data 



if.O 



3.3 



1.7 



3.3 



.6 



2.6 



3.1 



18.8 




Jbcores per cent 



8C^ 



81% 



5^ 



1% 



if3^ 



7B% 



5% 
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III. SCIEKCaB F50CESSES 

The evaluation conducted in the area of science processes differs 
substantially from the original evaluation design contained in the continuation 
application* Tbie became necessary as a result of the unavailibility of 
Dr« David Butts, the science consultant who was to provide the workshop 
e3q)erti6e, test forms, and raw data analysis • Vben it became evident that 
Sr. Butts was not going to be able to spend the amount of time required to 
accomplish these responsibilities « a different approach was followed* 

It was felt by the secondary science department that their curriculum 
lacked continuity between different grade levels; did not provide enough 
diflterentiation of instzoiction to accomodate all the students ensrolled in 
the science program; did not place sufficient emphasis upon the processes 
of science and finally, did not utilize all available instructional resources. 

Science workshops which provided teachers with the knowledge 
necessary to improve their curriculum were offered* These workshops utilized 
area consultants who were recognized leaders in their particular fields 
of interest. The culminating activi*^ in each workshop was the development 
of a curriculum guide. These guides were then evaluated by comparing them 
to an evaluation form developed expreeeHy for this pirpose« The data from 
the evaluation is presented in the table below* 



TABLE IV 

Scores on Science Curriculum Guide Check List 





Section 


Total 

Points 

Scored 


% of 
Total Points 
Possible (100) 


Grade Level 


I 


II 


III 


IV 


Seventh 

Eighth 

Nine-Twelve 


20 
25 


15 
25 
25 


25 
20 

25 


0 
5 
5 


60 

75 
80 


609^ 
7596 


Mean Scores 


23 


22 


23 


3 


72 




Total Points 
Possible 

r 


25 


25 


25 


25 


100 


1005^ 
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Table V contaizie supportive data for objective V 
Science Processes* The objective stated that 25 of 35 particlpatizig 
teachers would use one of three ^^process approach** science units in their 



dassroon instruction: AAAS, SCIS, or ESS 

TABLE V 

Participants Using l^ocess Approach Science Programs 



No. of 
Participants 


Barticipante Using 
Prior to Vforkshop 


Participants Using 
After Workshop 


Criteria 
Met 


No. % 


No, % 




6 12?5 


37 7396 


Yes 



IV. SOCIAL sTODiEs vmm 

The original evaluation design was modified somewhat so that a 
more accurate evaluation v^f the 1972*73 i^rkshop activities would be possible* 
Workshop participant' % sveloped detailed curzlculum guides which incorporated 
all the objectives listed in Section VI of Project Activities in the 1972-73 
Continuation Application. The curriculum guides were then evaluated using 
a specially prepared curriculum guide check list* The results of this 
evaluation are presented below* 

TABZ£ VZ 

Scores on Social Studies Oorriculum Guide Chei;^ List 



Quide Grade 
Level 


Section 


Total 

Points 

Scored 


% of 
Total Points 
Possible 


Criteria 

Met 
(Yes-N6) 


I 


1 II 


ni 


IV 


Seventh 


30 


20 


30 


20 


100 


100?S 


Tes 


Si^th 




20 


30 


15 


95 


9556 


Yes 


Ninth 


15 


20 


25 


20 


80 


S09^ 


Yes 


Mean Scores 


25 


20 


28 


18 


91 


9156 


Yes 


Total Points 
Possible 


30 

1 


20 


30 


20 


100 

1 


. 1005IS 
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Chapter III 



FINDINGS AND CJONCLUSIONS 

The 1972-75 Title III project had eevon major objectives which 
were used as a focus for workshop activities^ All of these objectives were 
met to some degree. Sejiarate workshops to accomplish the objectives of 
Effective Questioning , Diagnosis and Evaluation and Student Behavior Mod- 
ification were not conducted^ due either to a lack of teacher interest, or 
because it was not appropriate. The objectives were accomplished in part,^ 
however I through the activities in the Individualized Instruction, Behavioral 
Objectives, Science and Social Studies workshops. 

FINDINGS 

There were seven major objectives which the project hoped to 
accomplish. The findings associated with each of these objectives are 
presented below. 

1. Teacher skill in writing behavioral objectives increased 
substantially as a result of their workshop experiences. 

2. Participating teo.chers gained sufficient knowledge and 
confidence to individualize at least one subject in 
their classroom. 

3. As a result of their e:3q)eriences in one or more workshops, 
teachers demonstrated sufficient awareness of, and 
competence in, diagnosing and evaluating individual 
learner needs. 

km The participants in the science workshops demonstrated 
competency in providing instruction in science process 
skills. 
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The inquiry/ concept/value approach to the teaching of 
social studies was adopted by the entire secondary social 
studies department. Their skill in recognizing and 
utilizing this approach was evident from on analysis 
of the curriculum guides which were developed in the 
workshop. 

Effective Questioning was not taught nor evaluated 
as a separate objective. An explanation of the progress 
toward the accorapLishmont of this objective will be given 
in the section entitled "conclusion". 

Student Behavior Modification was not measured because 
existing circumstances precluded the establishment of a 
statistically valid testing situation* 
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CONCLUSIONS 

It is imperative to the proper interpretation of this evaluation 
data that the reader understand the conditions under which the raw data 
was collected and analysed* 

This evaluation did not contain a sophisticated evaluation design 
which attempted to control and manipulate all possible intervenizig variables* 
The raw data has been analyzed using pre and post tests t check lists and 
observation forms* A comparison was made between the raw data and the project 
objectives and results have been reported in terms of success or lack of 
success in meeting the stazidards stated in each objective* 

OBJECTIVE I - EFFECTIVE QI3ESTI0NING 

Separate workshops in effective questioning were not offered 
in 1972-73* A workshop was discussed with teachers but they felt the 
primary objective of decreasing "teacher talk" co\ild be satisfactorily 
accomplished through the activities contained in other workshops* 

The objective stated that 50 of 70 (703^) participating teachers 
would decrease by 2^ the amount of time spent in teacher talk* 

There are several indications that progress toward meeting this 
objective was made* Sections I* IV and VII on the Individualized Instruction 
Obsex^ation Check List (IIOC) encompasses teacher behavior whicht if practiced 
in the classroom « woxxld lead to a decrease in "teacher talk"* Tables II and 
III show that all participating teachers in the Individualized Instruction 
workshop scored above 709$ in these three categoxdes* Sections I and III 
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on the Science Curriculum Guide Check List (Table IV) also relates to teacher 
behavior which would decrease ''teacher talk"* The mean per cent in each 
of these categories is above 

Copies of both the HOC and the Science Curricultun Guide Check 
List can be found in the Appendix* 

OBJECTIVE II - BEHAVIORAL OBJECTIVES 

The workshops which provided participants with an opportunity to 
write objectives in behavioral terms were successful in increasing teacher 
skill in this area as evidenced by the data recorded in Table I* 

OBJECTIVE III - DIAGNOSIS & EVALUATION 

It was assumed that as a resi&lt of the workshops offered during 
1972-73 that participants would become more sldllful in diagnosing individxial 
learner need6« 

There was no direct measurement of the progress made toward the 
accomplishment of this objective* However 1 an analysis of Section II of 
the HOC (tables II & III) indicates that teachers were including diagnosis 
and evaluation as part of their planning for individualized instruction* 

Additional data to support the accomplishment of this objective 
is derived from the fact that all thirty teachers enrolled in the Croft 
Beading workshop were required « after sufficient trainizig« to administer 
and score the Cooper-McGuire Diagnostic Beading Test* Based upon the results 
of this testy teachers then prescribed individual reading programs for 
each student* 
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OBJECTIVE IV - INDIVIDUALIZED INSTRUCTION 

Tables II and III lend supportive data to the conclusion that 
objective IV - Individualized Instruction was met. 

Twenty-one out of twenty-two participating teachers satisfactorily 
implemented an individualised instructional program in at least one subject 
area* This exceeds the standard set forth in the objective* 

OBJECTIVE V - SCIEWCES PROCESSES 

Tables IV and V indicate that: (1) seventy three per cent of 
participating teachers are using a process approach in their science instruction 
and (2) provision has been made for the teaching of science processes in 
the new secondary curriculum guides* 

A competency measure was not given to participating teachers ^ 
therefore, their competency in teaching the science processes is n^rely 
an assumption. 

OBJECTIVE VI - SOCIAL STDDIES INQUIRY 

Section III of Table VI indicates that this objective was at least 
partially met. Teachers have made provisions in the newly developed curriculum 
guides for student oriented, inqtiiry instruction* Further, a check of the 
curriculum guides reveals that materials and activities listed are clearly 
inquiry/concept/value oriented* 



16 



OBJECTIVE VII - STDDENT BEHAVIOR MODIFICAnON 

There is no supportive data to indicate that student behavior 
was modified as a result of the in-service activities. As esqaLained earlier 
in this evaluation report y this was not measured because the existing 
circumstances did not permit a valid testing situation. The data from 
tables III III, IV, V and VI give credence to the assumption, however, that 
there was a degree of student behavior modification as a result of the 
modified teacher behaviors. 

SDMMAIiY 

The data contained in this evaluation report indicate that progress 
was made in the acconqplishment of eadi objective* There was, however, a 
lack of objective data to substantiate the contention that all objectives 
were met. 

Objectives II - Behavioral Objectives; IV - Individualized Instruction 
V Science Processes and VI - Social Studies Inquiry weare all supported by 

■y 

objective data which revealed either total or partial goal achievement. 
Objectives I - Effective Questioning; III - Diagnosis 8e Evaluation, and; 
VII Student Behavior Modification were not supported by a direct measurement. 
A subjective evaluation, arrived at through an analysis of the evaluation 
data, conversations with workshop participants, a study of teacher prepared 
materials and classroom observations, indicates that these objectives were 
also accomplished wholly or in part. 
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Francis Howell School District - ESEA Title III 
Individualized Instruction Observation Checklist 



Im Differentiated Instruction 

A. Flexible time schedule 

Bo Subgroup activities (flexible grouping) 

C. Assignments of varying difficulty 

D. Assignments of varying length 

E. Student assignments correlated with ability 
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II • Diagnosis 

Ao Arrangements for periodic diagnosis 

B, Provide for both group and individual diagnosis 

C, Utilize both teacher and commercially prepared tests 

D, Utilize diagnostic information in planning instructional activities 

III. Evaluation I I 

A, Clearly stated objectives for each unit 

B. Student involvement in evaluation process 

C, Periodic "informal" evaluation 

D. Planned comprehensive evaluation 

XV. Instxmctional Activities « q*^ | | 

A, Activities planned to meet individual needs 
B« Activities structured in small gradations 
C« Activities are sequential 

D. Positive reinforcement 

E. Provision for individual student interests (Learning Centers, etc.) 

F. Immediate feedback to students on their progress in each 
instructional activity 

V. Teacher-Pupil Conferences • ..••••o [ [ 

A. Students help plan individual goals 
Bp Students help plan learning activities 

C, Students assist in evaluation of individual achievement 

D, Teacher hao definite schedule for teacher-pupil conferences 

VI. Record Keeping « | 

A. Pupil kept records 

1. Records are simple and easy to understand 

2. Student feels responsibility for keeping records accurate and 
current 

Teacher kept records 

1. Individual progress records 

2. Cumulative records 

3« Subject matter profile records 
ho Class profile sheet 

VII. Multimedia Lj 

A. Availibility •f audio visual equipment 
Bo Utilization of audio visual equipment 

C. Skills presented throu^ a variety of instructional materials 
(textbook 9 loops I films, etc.) 

D. Media method correlated with student need and ability 
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SCIENCE CDRRICDLW! GUIDE CHECK LIST 



I. Differentiated Instruction 

A. Assigninents vary in length 

B. Assignments vary in difficulty 

C. Utilization of special i^ojects 

D. Instruction and activities designed for small as 
well as large groups 

£• Flexible time arrangement for each study unit 



II. Multimedia 

A. Availibility of equipment listed in guide 

B. Opportunity for utilization of available equipment 

C. Student use of multi-media material 

D. Organization and listing of all necessary 
laboratory material 

£• Laboratory materials are p^^actical for the activities 



HI. Instructional Activities 

A. Laboratory activities are student oriented 

B. Activities follow a sequential pattern 

C. Lab work correlated with material covered in 
lecture 

D. Detailed laboratory instructions for individual 
work 

£• Provisions for individual student interest 
F. Adequate provisions for learning science process 
skills 



IV. Evaluation 

A. Clear objectives » overall and for each unit 

B. Frequent opportunity for evalxiation 
C« Sample test questions in each unit 
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SOCIAL snroiEs cusbicdlum guide check list 



I. Differentiated Instruction 

A. AesigruDents vary in length 

B. Assignments vaxy in diffictilty 

C. Provision for special projects 

D. Provision for individual student interest 

Em Instruction and activities designed for small 

as well as large groups 
F. Flexible time arrangement for each study unit 



II. Multimedia 

A. Availibility of equipment listed in guide 

B. Student use of multi-media material 

C. Media used is practical for the skill or concept 
being learned 

D. Skills or concepts are presented through a variety 

of instructional media (textbookst loops, films, etc.) 



III. Izistructional Activities 

A. Activities follow a sequential pattern 

B. Activities are student oriented 

C. Interdisciplinary approach is used in the 
presentation of concepts and skills 

D. Concepts are arrived at through student inquiry 

E. Student inquiry is guided and directed 

F. Provision for individual interests (Learning 
Centers, etc.) 



IV. Evaluation 

A. Clear objectives, overall and for each iinit 

B. Frequent opportunity for evaluation 

C. Student knowledge of evaluation resxilts 

D. Sample test questions contained in units 
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